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Abstract 

This article reviews published experimental studies from 2000 to 2012 that evaluated the 
effects of providing reading interventions to English learners who were at risk for experiencing 
academic difficulties, including students with learning disabilities. Criteria included: (a) the study 
was published in a peer-referred journal, (b) the study was an intervention for English learners 
at risk or with a learning disability in Grades K—I2, (c) data were disaggregated by English 
learner status if all participants were not English learners, and (d) information about fidelity 
of implementation was reported. Twelve studies met these criteria. Results of seven studies 
conducted in kindergarten and first grade indicated significant moderate-to-large effect sizes 
(ES range, 0.58-0.91) for interventions targeting beginning reading skills. Findings in five of the 
12 studies suggested significant moderate-to-large effects in reading or listening comprehension 
(ES range, 0.47—2.34). The interventions in these studies included explicit instruction, and 10 
used published intervention programs. Moderator variables, such as group size, minutes of 
intervention, and type of personnel delivering the intervention, were not significant predictors 
of outcomes. 


English learners remain the fastest-growing 
group of students in American schools, with 
large increases occurring in most regions of 
the country (U.S. Department of Education, 
2011). For a variety of reasons, including the 
fact that these students have to learn a second 
or even third language while also mastering 
grade-level content, they form a significant 
portion of students who struggle academi- 
cally. Only 7% of fourth-grade and 3% of 
eighth-grade English learners score at or 
above proficiency on reading assessments as 
compared to 38% and 37% of native English 
speakers (National Center for Educational 
Statistics, 2014). These data highlight the 
importance of instructional interventions to 
support their academic progress as well as 
their English language proficiency abilities. 
However, little is known about the additional 


support English learners receive in schools as 
part of a response-to-intervention model. 
Until recently, few published studies have 
described the effectiveness of interventions 
and support programs for English learners 
(Gersten & Baker, 2000; Klingner, Artiles, & 
Barletta, 2006). Although this situation has 
begun to improve in the past decade, with 
an increasing number of rigorous research 


‘California State University, Long Beach 
Southern Methodist University 
3Instructional Research Group 
‘University of Oregon 


Corresponding Author: 

Catherine Richards-Tutor, California State University, 
Long Beach, EDI-10 ASEC, 1250 Bellflower Blvd, Long 
Beach, CA 90840, USA. 

E-mail: Cara.Richards-Tutor@csulb.edu 


Downloaded from ecx.sagepub.com by guest on August 12, 2015 


Exceptional Children 


studies investigating the instructional effec- 
tiveness of interventions, results of these 
studies have not been synthesized in a cohe- 
sive manner for scientific purposes or to 
guide practitioners in their implementation 
of effective interventions for this population 
of students. In this article, we describe and 
synthesize this recent research. Our review 
includes summarizing findings from previ- 
ous reviews, manifestations of poor educa- 
tion and intervention support for English 
learners, and the disproportionate overrepre- 
sentation and underrepresentation of English 
learners in special education (August & 
Hakuta, 1997; August & Shanahan, 2006; 
Klingner et al., 2006). We also discuss the 
implications for both future research and 
practice. 


Past Syntheses on 
Interventions for English 
Learners 


In 2000, Gersten and Baker reviewed research 
on effective instructional practices for English 
learners. The authors found few experimental 
studies and consequently expanded their synthe- 
sis to include descriptive and qualitative studies 
and also incorporated the professional opinions 
of practitioners with specific expertise in work- 
ing effectively with English learners (1.e., Noblit 
& Hare, 1988; Ogawa & Malen, 1991). Studies 
reviewed included case study research, qualita- 
tive research, descriptive studies, and a small 
number of experiments and quasi-experiments. 
This analysis was not limited to interventions for 
struggling English learners in reading but 
included any research on K-8 instruction. 
Results of the synthesis suggested several prom- 
ising approaches for improving instruction for 
English learners: (a) using vocabulary as a cur- 
riculum anchor across multiple subject areas, 
(b) using graphic organizers and other physical 
artifacts to reinforce concept acquisition and 
growth in academic vocabulary, (c) using coop- 
erative and peer-tutoring strategies to enhance 
engagement and nonthreatening articulation and 
discussion of newly acquired content, (d) strate- 
gically using the native language when neces- 


sary, and (e) modulating cognitive and language 
demands depending on the lesson objectives. 

In a more recent review, Klingner et al. 
(2006) located eight studies conducted since 
1997 that met their criteria for credible 
approaches toward reading instruction for Eng- 
lish learners. The evidence base of this review 
centered on qualitative studies rather than exper- 
imental research. Only two of the eight studies 
reviewed were experimental, and one of the two 
was conducted in India. The authors articulated 
what they viewed as promising practices based 
on the studies reviewed. These promising prac- 
tices included (a) combining phonological 
awareness (PA) with other English language 
development activities, (b) teaching and encour- 
aging the use of reading comprehension strate- 
gies in the first and second language, (c) helping 
students develop a strong foundation in reading 
in both their native language and in English, and 
(d) heavy emphasis on rich vocabulary instruc- 
tion. In summary, findings from both syntheses 
converge in the importance of providing strong 
vocabulary instruction and native language sup- 
port. However, in neither the Gersten et al. 
(2000) nor the Klingner et al. (2006) syntheses 
was the evidence base sufficient to draw clear 
conclusions about best practices. Both were 
intended to serve as means for delineating prom- 
ising practices that could subsequently be evalu- 
ated with rigorous research. 

At the same time, guidance to practitioners 
regarding how to effectively teach reading to 
English learners remains a pressing national 
priority. In response to this demand, the Insti- 
tute of Education Sciences (IES) has published 
two practice guides (S. Baker et al., 2014; Ger- 
sten et al., 2007) intended to provide specific 
recommendations as to evidence-based prac- 
tices for teaching English learners and also to 
delineate areas where there is no solid evidence 
base but where expert opinion suggests specific 
ideas for best practice. The importance of 
instruction that builds academic language and 
academic vocabulary was stressed in both doc- 
uments. In addition, both guides concluded that 
there was solid empirical evidence for the use 
of explicit, small-group instruction to improve 
the reading proficiency of struggling English 
learners. Explicit instruction provides the 
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necessary scaffolds students need to under- 
stand the concepts taught. Small-group instruc- 
tion provides English learners with extended 
opportunities to use English and multiple 
opportunities to interact closely with the 
teacher. 

Moreover, skilled teachers can take advan- 
tage of small-group instructional opportuni- 
ties to not only provide targeted and modulated 
instruction to meet the precise needs of indi- 
vidual children but also provide additional 
opportunities for English learners to speak, 
hear, and read English. These additional 
opportunities may help English learners 
develop their English language proficiency, 
an important component of comprehension 
(D. Baker, Park, & Baker, 2013; Farnia & 
Geva, 2011; Gottardo & Mueller, 2009). 

In this article, we extend the findings of 
previous reviews and the two IES practice 
guides by calculating the measurable impacts 
of interventions from published experimental 
studies that have been conducted since 2000 
with English learners identified as at risk or 
with learning disabilities. We included only 
experimental studies (i.e., randomized control 
trials [RCTs]) because we were interested in 
determining if there were practices with 
causal evidence of effectiveness that had been 
subjected to the peer-review process and 
could be more easily accessed by practitio- 
ners. To determine which studies to include, 
we followed procedures identified by the 
What Works Clearinghouse (2014) to deter- 
mine the quality of the methodology. Specifi- 
cally, we attempted to do the following: 


1. Summarize the specific study features 
and intervention characteristics, includ- 
ing (a) group size, (b) duration, (c) per- 
sonnel delivering the instruction, (d) 
intervention content, (e) intervention 
method, and (f) the counterfactual, that 
is, the nature of instruction delivered to 
the control group. 

2. Calculate the impact of the interven- 
tions on core components of reading, 
including (a) PA, (b) word reading, (c) 
passage reading fluency, (d) vocabu- 
lary and oral language, and (e) reading 


comprehension, including reading 
cloze measures and listening compre- 
hension measures. 

3. Explore the effects of specific moder- 
ator variables: group size, duration, 
and personnel delivering instruction 
on student outcomes. 


Method 


We used the following criteria to identify RCT 
studies in peer-reviewed journals for review: 
(a) The study sample comprised English 
learners in kindergarten through 12th grade 
who were identified as at risk or with a learn- 
ing disability (using either standardized tests 
or valid screening measures), (b) data were 
disaggregated by English learner status if not 
all participants were English learners, and (c) 
information about fidelity of implementation 
was reported. These criteria were used because 
they allowed for precise impact estimates to 
be calculated for English learners specifically 
and helped ensure how interventions were 
delivered and the extent to which delivery 
matched study expectations. After we identi- 
fied studies that fit these criteria, we used the 
WWC procedures to calculate the effect sizes 
and summarize these effects in terms of mean- 
ingful categorizations. 

During January through March 2013, the 
search focused on studies from 2000 to 2012 in 
PsycInfo and ERIC, using the following key- 
words individually in peer-reviewed journals: 
English learners, language minority students, 
second language learners, _ intervention, 
response to intervention, at-risk, learning dis- 
abilities, reading difficulty, writing difficulty, 
and math difficulty. We then specifically 
searched for studies in the following journals: 
Journal of Educational Psychology, Journal of 
Learning Disability, Learning Disabilities 
Research & Practice, Learning Disabilities 
Quarterly, Reading Research Quarterly, Reme- 
dial and Special Education, Scientific Studies of 
Reading, The Journal of Special Education, 
Exceptional Children, Journal of Literacy 
Research, and Topics in Language Disorders. 
Once we identified potential studies, we read the 
abstracts and selected for further analysis only 
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those studies that indicated they used an RCT 
design and included struggling K-12 English 
learners as participants (i.c., English learners 
who were receiving a Tier 2 or Tier 3 interven- 
tion). Next, we carefully reviewed the Method 
section of each article to ensure that the studies 
met our other criteria. We located three studies 
that met these criteria but used single-case 
design methods. We do not reference these stud- 
ies in this review, but we published a separate 
technical report on these three studies (see Rich- 
ards-Tutor, Baker, Gersten, Baker, & Smith, 
2014). 

Once we identified eligible studies, we used a 
coding form to summarize the information from 
each study by two broad categories: (a) features 
of the research study (1.¢., research design, grade 
level, participant characteristics, setting) and (b) 
characteristics of the interventions (i.e., group 
size, duration, personnel delivering the interven- 
tion, intervention content, intervention methods). 
Table | presents the features of the research stud- 
ies, and Table 2 presents the characteristics of the 
interventions. Two raters independently coded 
each of the studies, and agreement between the 
raters was 90% or above for each of the features. 
Disagreements were resolved through discus- 
sions between the two raters. 

Our standards were similar to—but not iden- 
tical to—those used by WWC (http://ies.ed.gov/ 
ncee/wwe/documentsum.aspx?sid=19). The only 
difference was that we did not exclude studies 
that demonstrated differential attrition as defined 
by WWC (2014, pp. 11-14). We made this deci- 
sion about attribution in part because the WWC 
approach is not currently a commonly used stan- 
dard for special education research. 

We calculated effect sizes for the English 
learner sample using Hedges’ g as suggested by 
the WWC (2014, p. 22) to ensure that all the 
effect sizes in the studies could be interpreted in 
a similar way (some of the studies also calcu- 
lated effect sizes using accepted procedures, 
such as Cohen’s d, but we conducted indepen- 
dent calculations for all studies and effect sizes). 
Hedges’ g is commonly used and it also corrects 
for potential error due to small sample size, and 
many of the studies included in this review had 
small sample sizes. In most cases, data from the 
published studies were sufficient to determine 


effect sizes. In the few cases where the data for 
effect size calculations were not available in the 
publication, we contacted the authors to obtain 
the necessary information. For two of the studies 
previously reviewed by the WWC, we requested 
and received the original WWC analyses of 
effect sizes for the English learner subsample. 


Calculation of Effect Size 


To calculate the effect size, we used an 
adjusted mean difference in the numerator and 
the pooled unadjusted standard deviation in 
the denominator. If adjusted means were not 
reported, we calculated effect sizes using a 
difference-in-differences approach to calcu- 
late the numerator (i.e., we computed a gain 
score for both experimental and control 
groups) and the pooled posttest standard devi- 
ation for the denominator. In addition, we cor- 
rected alpha levels to account for multiple 
comparisons (i.e., multiple measures assess- 
ing the same outcome) using the Benjamini- 
Hochberg procedure. We set alpha at .05 for 
each reading or prereading domain (e.g., PA, 
reading fluency, comprehension, vocabulary). 
To analyze potential moderating variables, we 
conducted three separate regression analyses 
(i.e., for group size, intervention duration, and 
personnel delivering the intervention) using 
the unweighted average effect size across all 
domains of reading included in each study. 


Results 


The comprehensive literature search yielded 
12 studies; all addressed reading or preread- 
ing skills (i.e., there were no studies in other 
domains, such as mathematics, science, or 
writing). We describe characteristics of the set 
of studies followed by findings organized by 
objective. 


Features of Research Studies 


Table 1 provides specific details regarding 
features of the intervention studies. The sample 
sizes for English learners in the studies ranged 
from 35 to 158. Multiple grade levels were 
included, although half of the studies were 
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Table |. Study Features. 


Grade Sample 
Authors level size EL primary language EL determination Reading risk determination 
O’Connor, Bocian, Beebe- K 35 Spanish CELDT DIBELS (LNF < 9 and ISF < 7) 
Frankenberger, & Linklater PPVT (SS < 85) 
(2010) 
Solari & Gerber (2008) K 82 Spanish Parent survey PA 
Vocabulary (criteria not reported) 
Vadasy & Sanders (2010) K 84 28 different languages, Parent survey In bottom half of class on LN, LS (Fuchs et 
including al., 2001), and CTOPP Sound Matching 
Spanish = 49% (composite-score z score) 
Vietnamese = 15% 
Somali = 6% 
Chinese = 6% 
Tagalog = 3% 
Gunn, Biglan, Smolkowski, & K-3 122 Spanish Not reported DIBELS composite: both some risk and high risk 
Ary (2000) categories 
Vaughn, Linan- Thompson, | 69 Spanish School WLPB LWID (below 25th percentile) 
et al. (2006) determination Experimenter word reading list (raw score < 2) 
Vaughn, Mathes, et al. (2006) | 48 Spanish School WLPB LWID (below 25th percentile) 
determination Experimenter word reading list (raw score < 2) 
Vaughn, Cirino, et al. (2006) | 190 total Spanish School WLPB LWID (below 25th percentile) 


Begeny, Ross, Greene, 
Mitchell, & Whitehouse 
(2012) 

Denton, Anthony, Parker, & 
Hasrouck (2004) 


(94 in Spanish intervention, 
96 in English intervention) 
2 2l Spanish 


2-5 93 total (n = 22, Grade 2;n Spanish 
= 37, Grade 3; 
n= 28, Grade 4; n = 6, 
Grade 5) 


determination 


School 
determination 


LAS 


Researcher developed word reading list (raw 
score < 2) 

GORT fluency or comprehension < 10 (SS 
mean) 


Teacher recommendation 
WRMT: LWID and WA subtests 
< Grade | 
equivalent = emerging decoding group 
WRMT: LWID and WA subtests > Grade | 
equivalent = established decoding group 


(continued) 
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Table |. (continued) 


Grade Sample 
Authors level size EL primary language EL determination Reading risk determination 
Lovett et al. (2008) 2-8 76 Multiple languages, including Parent survey Teacher referral 
Portuguese = 49% WRMT: LWID and WA (SS < 85) 
Spanish = 21% WRAT-3 (SS < 85) 
Wanzek & Roberts (2012) 4 74 Spanish School Teacher referral 
determination GMRT (SS < 25th percentile) 
Vaughn et al. (2011) 7-8 42 Spanish Not reported TAKS (SS < 30th percentile) 


Note. EL = English learner; CELDT = California English Language Development Test; DIBELS = Dynamic Indicators of Basic Early Literacy Skills, 6th ed. (Good & Kaminski, 2002); LNF = 
Letter Naming Fluency; ISF = Initial Sound Fluency; PPVT = Peabody Picture Vocabulary Test (Dunn & Dunn, 1981); PA = phonological awareness; LN = letter naming; LS = letter sounds; 
CTOPP = Comprehensive Test of Phonological Processing (Wagner, Torgesen, & Rashotte, 1999); WWLPB = Woodcock Language Proficiency Battery (Woodcock, 1991); LWID = Letter 
Word Identification; WA = Word Attack; GORT = Gray Oral Reading Test (Bryant, Shih, & Bryant, 2009); WRMT = Woodcock Reading Mastery Test (Woodcock, 1987); LAS = Language 
Assessment Scales (De Avila & Duncan, 1990); WRAT = Wide Range Achievement Test (Wilkinson, 1993); GMRT = Gates-MacGinitie Reading Test (MacGinitie, MacGinitie, Dreyer, & 
Hughs, 2006); TAKS = Texas Assessment of Knowledge and Skills; SS = standard score. 
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conducted in kindergarten or first grade. One 
study involved second-grade students (Beg- 
eny, Ross, Greene, Mitchell, & Whitehouse, 
2012) and one study involved fourth graders 
(Wanzek & Roberts, 2012). Two studies 
involved students in the upper elementary 
grades (Denton, Anthony, Parker, & Hasrouck, 
2004; Gunn, Biglan, Smolkowski, & Ary, 
2000), and two studies involved students at the 
middle school level (Lovett et al., 2008; 
Vaughn et al., 2011). 

Half the studies included English learners 
only (Begeny et al., 2012; Denton et al., 2004; 
Solari & Gerber, 2008; Vaughn, Cirino, et al., 
2006; Vaughn, Linan-Thompson, et al., 2006; 
Vaughn, Mathes, et al., 2006); the remaining six 
included both English learners and native Eng- 
lish speakers, but we were able to separately 
analyze the English learner subsample. In all 
but two of the studies, participants were from 
homes where Spanish was the primary home 
language. In the two other studies (Vadasy & 
Sanders, 2010; Lovett et al., 2008) the English 
learner participant sample included numerous 
languages. 


Methods used to determine English learner sta- 
tus. The majority of studies (n = 7) used as 
their criteria the school designation of English 
learners (Begeny et al., 2012; Denton et al., 
2004; O’Connor, Bocian, Beebe-Franken- 
berger, & Linklater, 2010; Vaughn, Cirino, 
et al., 2006; Vaughn, Linan-Thompson, et al., 
2006; Vaughn, Mathes, et al., 2006; Wanzek & 
Roberts, 2012). Three studies used parent 
questionnaires or interviews to determine the 
primary language spoken at home (Lovett 
et al., 2008; Solari & Gerber, 2008; Vadasy & 
Sanders, 2010), and two studies did not report 
specifically how English learner status was 
determined (Gunn et al., 2000; Vaughn et al., 
2011). 


Methods used to identify risk status and learning 
disability. Three studies included English learn- 
ers with identified learning disabilities (Lovett 
et al., 2008; Vaughn et al., 2011; Wanzek & 
Roberts, 2012) in their sample. The remaining 
studies included students deemed at risk for 
learning disabilities, but definitions of how risk 


was determined varied. For the six kindergarten 
and first-grade studies, risk determination typi- 
cally included screening measures of PA, alpha- 
betic knowledge, or word or pseudoword 
reading. In contrast, both Solari and Gerber 
(2008) and O’Connor et al. (2010) used the Pea- 
body Picture Vocabulary Test (Dunn & Dunn, 
1981) to determine risk status. The three first- 
grade studies by Vaughn et al. (Vaughn, Cirino, 
et al., 2006; Vaughn, Linan-Thompson, et al., 
2006; Vaughn, Mathes, et al., 2006) and the 
Solari and Gerber study assessed students both 
in their native language (1.e., Spanish) and in 
English as part of the risk determination pro- 
cess. In the studies that targeted students in sec- 
ond grade and above, all used standardized 
reading achievement tests for risk determina- 
tion, and three (Denton et al., 2004; Lovett 
et al., 2008; Wanzek & Roberts, 2012) also took 
teacher recommendation into account in deter- 
mining who was at risk. 


Intervention Characteristics 


The intensity of an intervention can be deter- 
mined by many characteristics, but three are 
common: group size, duration of the interven- 
tion, and quality of the personnel delivering 
the intervention and the associated amount of 
training they receive (Gersten et al., 2008). We 
describe these aspects of the studies next, and 
to provide a more complete picture of the type 
of instruction students received, we also 
describe the content and methods used to 
deliver the intervention and provide details 
regarding information on the counterfactual 
(see Table 2). 


Group size. Two studies (Begeny et al., 2012; 
Vadasy & Sanders, 2010) used one-on-one 
tutoring. The remainder used relatively small 
homogeneous groups of students who read at 
similar levels of proficiency. Half the studies 
included groups of three to five, two studies 
used even smaller groups, and one study 
(Lovett et al., 2008) included groups as large 
as eight students. 


Duration of the intervention. Intervention ses- 
sions ranged from 20 to 60 min. The shortest 
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Table 2. Intervention Characteristics. 


Group size and 


Authors composition Intervention duration Interventionist Intervention program Intervention content 
O’Connor, 2-3; 36 weeks, 3 days per week, 15 min = Paraeducators Ladders to Literacy Alphabetics, phonological 
Bocian, Beebe- homogeneous __ per day (270—1,430 min) awareness, oral language 
Frankenberger, & groups 
Linklater (2010) 
Solari & Gerber 4-5; 8 weeks, 3 days per week, 20 min Research assistants NA Phonological awareness, 
(2008) homogeneous __ per day (480 min) listening comprehension 
groups 
Vadasy & Sanders One-on-one 18 weeks, 4 days per week, Paraeducators NA Alphabetics, phonological 
(2010) instruction 30 min per day (2,160 min) awareness, word reading, 
spelling, oral language 
Gunn, Biglan, 2-3; 60 weeks, 5 days per week, 25-30 Paraeducators Reading Mastery and Phonological awareness, 
Smolkowski, & homogeneous min per day (7,500—9,000 min) Corrective Reading alphabetics, reading fluency 
Ary (2000) groups 
Vaughn, Linan- 3-5; 32 weeks, 5 days per week, 50 min Bilingual certified teachers Lectura Proactiva Letter knowledge, word 
Thompson, homogeneous _ per day: (4,560—6,900 min) hired by research team recognition, fluency, 
et al. (2006) groups comprehension, oral language 
skill, vocabulary 
Vaughn, Mathes, 3-5; 32 weeks, 5 days per week, 40 min Bilingual certified teachers Proactive Reading Letter knowledge, word 
et al. (2006) homogeneous __ per day (6,400 min) hired by research team recognition, fluency, 
groups comprehension, oracy, 
vocabulary 
Vaughn, Cirino, 3-5; 32 weeks, 5 days per week, 50 min Bilingual certified teachers Lectura Proactiva or Letter knowledge, word 
et al. (2006) homogeneous __ per day (4,476—6,402 min) hired by research team Proactive Reading recognition, fluency, 
groups comprehension, oracy, 


Begeny, Ross, 


Greene, Mitchell, & 
Whitehouse (2012) 


One-on-one 
instruction 


20-28 weeks, 2—3 times per week, 


10 min per day (600-840 min) 


Lead researcher, graduate 
and undergraduate 
students 


HELPS Fluency 
Program 


vocabulary 
Reading fluency, comprehension 


(continued) 
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Table 2. (continued) 


Authors 


Group size and 


composition Intervention duration 


Interventionist 


Intervention program 


Intervention content 


Denton, Anthony, 
Parker, & 
Hasrouck (2004) 


Lovett et al. (2008) 


Wanzek & 
Roberts (2012) 
Vaughn et al. (2011) 


I-4; 10 weeks, 3 days per week, 
homogeneous 40 min per day (1,200 min) 
based on 
decoding 

4-8; 21 weeks’, 4-5 days per week, 
homogeneous 60 min per day (6,300 min) 


groups based 
on decoding 


2-4; within 28 weeks, 5 days per week, 
schools 30 min per day (2,550—3,420 min) 
4-5 (not 32 weeks, 5 days per week, 
reported) 50 min per day (8,000 min) 


Read Well or Read 
Naturally 


Undergraduate students 


Certified special education Reading Mastery or 
teachers Corrective 
Reading 


Certified teachers hired 
by research team 

Certified teachers hired 
by research team 


Wilson Reading 


REWARDS and 
Wilson Reading 


Alphabetics, reading fluency, 
vocabulary, comprehension 


Alphabetics, word reading, 
phonological awareness 
Word reading, comprehension 


Reading fluency, vocabulary, 
comprehension 


Note. The instructional program and content in the control condition was whatever the district and school typically provided to students. Homogenous refers to academic levels and 
not English proficiency level or English learner status. 
“Not reported, calculated by authors. 
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sessions were the kindergarten studies with 
10- to 20-min sessions, whereas most other 
sessions ranged from 30 to 60 min. Six studies 
included daily intervention sessions; the 
remainder varied from twice a week to four 
times a week. Length of intervention also var- 
ied substantially. These factors yielded a large 
range in the total number of minutes of inter- 
vention provided, with the range being from 
270 min to 9,000 min. The average interven- 
tion was approximately 3,600 min in duration, 
which equals to about 120 thirty-minute 
lessons. 


Personnel delivering instruction. Teachers deliv- 
ered the instruction in six of the interventions. 
In three studies, teachers with specific train- 
ing in bilingual education delivered the inter- 
ventions (Vaughn, Cirino, et al., 2006; 
Vaughn, Linan-Thompson, et al., 2006; and 
Vaughn, Mathes, et al., 2006), and in another, 
special education teachers delivered the inter- 
vention (Lovett et al., 2008). Two studies 
hired outside teachers to deliver the interven- 
tions (Vaughn et al., 2011; Wanzek & Roberts, 
2012). Three studies employed paraprofes- 
sionals (Gunn et al., 2000; O’Connor et al., 
2010; Vadasy & Sanders, 2010), and in three 
other studies, research assistants, undergradu- 
ates, or graduate students delivered the inter- 
vention (Begeny et al., 2012; Denton et al., 
2004; Solari & Gerber, 2008). In all cases, 
teachers, paraprofessionals, or research per- 
sonnel were trained on how to deliver the 
intervention; they were observed regularly; 
and they were provided with feedback. 


Content of the interventions. Half the studies 
(Denton et al., 2004; Vaughn, Cirino, et al., 
2006; Vaughn, Linan-Thompson, et al., 2006; 
Vaughn, Mathes, et al., 2006; Vaughn et al., 
2011; Wanzek & Roberts, 2012) used a com- 
prehensive intervention that covered at least 
four of the five areas of literacy outlined in the 
National Reading Panel (Ehri et al., 2001) and 
the National Literacy Panel for Language 
Minority Students (August & Shanahan, 
2006): phonemic awareness, phonics, fluency, 
vocabulary, and comprehension. The other 
half (Begeny et al., 2012; Gunn et al., 2000; 


Lovett et al., 2008; O’Connor et al., 2010; 
Solari & Gerber, 2008; Vadasy & Sanders, 
2010) focused on just one or two components 
of reading. In general, studies that targeted the 
kindergarten level (e.g., O’Connor et al., 
2010) focused on PA and alphabetic knowl- 
edge, and studies that targeted the intermedi- 
ate grades (e.g., Begeny et al., 2012) focused 
on fluency and comprehension. 

Five of the studies included vocabulary as 
one of the proficiencies (Denton et al., 2004; 
Vaughn, Cirino, et al., 2006; Vaughn, Linan- 
Thompson, et al., 2006; Vaughn, Mathes, 
et al., 2006; Vaughn et al., 2011), and four 
studies focused on oral language development 
as a key skill targeted in the intervention 
(O’Connor et al., 2010; Vaughn, Cirino, et al., 
2006; Vaughn, Linan-Thompson, et al., 2006; 
Vaughn, Mathes, et al., 2006). None of the 
studies included building of academic vocab- 
ulary or academic language as an explicit 
goal, although we suspect that those with an 
oral language or vocabulary component prob- 
ably did address these topics to some extent. 

Ten studies used existing curricula in the 
treatment condition, including Reading Mas- 
tery (Engelmann & Bruner, 1995), Corrective 
Reading (Engelmann, 1988), Read Well 
(Sprick, Howard, & Fidanque, 1998), Read 
Naturally (Ihnot, Mastoff, Gavin, & Hendrick- 
son, 1992), Ladders to Literacy (O’Connor, 
Notari-Syverson, & Vadasy, 2005), HELPS 
(Begeny, 2009), Wilson Reading System (Wil- 
son, 2002), REWARDS (Archer, Gleason, & 
Vachon, 2000), or modified versions of exist- 
ing curricula (1.e., Proactive Reading; Mathes, 
Menchetti, Wahi, & Grek, 2004). Two studies 
used a combination of existing curricula and 
researcher-developed curricula, and the deci- 
sion about which materials to use was based on 
the skills taught (Vaughn et al., 2011; Wanzek 
& Roberts, 2012). Two studies developed and 
tested novel interventions in the treatment con- 
dition (Solari & Gerber, 2008; Vadasy & Sand- 
ers, 2010). Note that most of these curricula 
were developed for use with the general popu- 
lation rather than as specialized curricula for 
English learners. 

In five of the studies, multiple interven- 
tions were tested in different treatment groups 
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(Denton et al., 2004; O’Connor et al., 2010; 
Solari & Gerber, 2008; Vaughn et al., 2011; 
Wanzek & Roberts, 2012). Only two studies, 
as indicated in Table 2, included interventions 
provided in the students’ primary language, in 
both cases Spanish (Vaughn, Cirino, et al., 
2006; Vaughn, Linan-Thompson, et al., 2006). 


Methods of intervention delivery. The use of sys- 
tematic, explicit instruction is the best way to 
describe the treatment intervention across all 
studies. Common instructional procedures 
included modeling, scaffolding, and corrective 
feedback. Three studies described features of 
intervention delivery that were designed spe- 
cifically to meet the needs of English learners 
learning to read in a relatively new language 
(Denton et al., 2004; Vaughn, Cirino, et al., 
2006; Vaughn, Mathes, et al., 2006). These 
features included using visuals and gestures, 
building background knowledge or activating 
prior knowledge, clarifying meanings of 
words, and showing differences between Eng- 
lish and the students’ primary language. 


The counterfactual: Nature of comparison group 
intervention and instruction. Of the 12 studies, 
11 described the control group instruction as 
“business as usual” or “typical practice.” The 
majority of studies (n = 10) provided some 
information about the control condition; two 
studies did not (Gunn et al., 2000; Vaughn 
et al., 2011). Three studies reported that the 
control group received the school’s core read- 
ing program, Tier | instruction (Begeny et al., 
2012; O’Connor et al., 2010; Vadasy & Sand- 
ers, 2010). Four studies reported that control 
students received supplemental intervention 
from their school in addition to Tier 1 (Denton 
et al., 2004; Vaughn, Cirino, et al., 2006; 
Vaughn, Linan-Thompson, et al., 2006; 
Vaughn, Mathes, et al., 2006). In the Denton 
study, seven of the students in the control 
group received supplemental intervention for 
60 to 240 min per week, and four received spe- 
cial education services for 60 min per week. In 
the Vaughn, Linan-Thompson, et al. (2006) 
study, 14 control students received interven- 
tion for an average of 3,822 min. In the Vaughn, 
Mathes, et al. (2006) study, 29 students 


received on average 5,040 min of supplemen- 
tal intervention. In the Vaughn, Cirino, et al. 
(2006) study, 27 students in the Spanish con- 
trol group received on average 2,472 min of 
intervention, and 28 students in the English 
intervention received 5,256 min of 
intervention. 

Two studies reported very specific informa- 
tion regarding the control group (Lovett 
et al., 2008; Wanzek & Roberts, 2012). In the 
Lovett et al. (2008) study, students in the con- 
trol group received the school’s typical special 
education language arts program. This pro- 
gram varied across schools and was locally 
developed. The students received the same 
number of minutes of instruction (6,300), and 
the intervention was conducted in similar 
group sizes (two to eight students). Wanzek 
and Roberts (2012) reported that eight of the 
control students received one supplemental 
intervention, and three received two supple- 
mental interventions. Nine of these students 
received 200 to 360 min of intervention per 
week, and two received 25 to 60 min of inter- 
vention per week. These interventions typi- 
cally took place in groups of two to three 
students, and the program focused on test-tak- 
ing skills. One of the studies (Solari & Gerber, 
2008) used an alternative treatment condition. 
The alternative treatment condition focused 
only on PA skills and was delivered in the same 
group size and for the same number of minutes 
as the two intervention conditions. 


Intervention Outcomes 


Table 3 presents the outcomes for English 
learners in each study, summarized across 
seven domains. Although the National Read- 
ing Panel (Ehri et al., 2001) suggested only 
five domains in reading, we decided to create 
three separate domains for comprehension 
because of research suggesting that compre- 
hension effects depend on how this component 
is measured (Cutting & Scarborough, 2006) 
and because for English learners, in particular, 
we thought the additional precision could be 
helpful in understanding intervention impact. 
Thus, we divided comprehension outcomes in 
the 12 studies into three categories based on 
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Table 3. Outcome Effect Sizes by Measurement Domain. 


Study 


PA 


Phonics/word 
reading 


Passage reading Vocabulary/oral 


fluency 


language 


Reading 
cloze 


Reading 
comprehension 


Ke 


O’Connor, 
Bocian, Beebe- 
Frankenberger, & 
Linklater (2010) 
Solari & Gerber 
(2008) 


Vadasy & Sanders 
(2010)? 


Gunn, Biglan, 
Smolkowski, & Ary 
(2000) 


Vaughn, Linan- 
Thompson, et al. 
(2006) Spanish 
intervention 


DIBELS PSF (g = .91**) 


LC concentration: 
Early PA®* (g = .63) 
Late PA* (g = -.14) 
PA concentration: 
Early PA* (g = -.57) 
Late PA* (g = -.74) 


CTOPP (g = .93**) 


TOPPS (g = .58*) 


LC concentration: 
WY] Ill LWID 

(g = -.19) 
WY] Ill WA 

(g = .43) 
PA concentration: 
WY] Ill LWID 

(g = .56) 
WY] Ill WA 

(g = -.07) 


WRMT LWID 
and WA‘ 
(g = .61**) 
WJ Ill LWID 
(g = .24) 
W] Ill WA 
(g = .52°*) 
WLPB-Spanish 
WA (g = .91**) 


Passage reading 
fluency* 
(g = .90**) 
DIBELS ORF 
(g = .24) 


IDEL ORF 
(g = .78**) 


WLPB-Spanish 
Picture 
Vocabulary 
(g = .28) 

WLPB-Spanish 
Verbal 
Analogies 


(g = .30) 


WRMT Passage 
Comprehension 
(g = .47°**) 


WLPB-Spanish 
Passage 
Comprehension 
(g = .88**) 


LC concentration: 

WY] Ill Story Retell 
(g = 2.34***) 

LC* (g = 1.73**) 

Domain average 
(g = 2.04***) 

PA concentration: 

WJ Ill Story Retell 
(g = 1.00*) 

LC* (g = 1.81**) 

Domain average 
(g = 1.41*) 


WLPB-Spanish LC 
(g = .50*) 


(continued) 
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Table 3. (continued) 


Phonics/word Passage reading Vocabulary/oral Reading Reading 

Study PA reading fluency language cloze comprehension LC 

Vaughn, Mathes, et = CTOPP (g = 1.24**) WLPB-English DIBELS ORF WLPB-English = WLPB-English — WLPB-English Listening 
al., (2006)? English WA (g = .69) (g = .18) Picture Passage Comprehension (g = 
intervention) Vocabulary (g¢ |= Comprehension .26) 

= .09) (g = .83*) 
WLPB-English 

Verbal 

Analogies (g = 

.78*) 
Domain average 

(g = .43) 

Vaughn, Cirino, et al. Spanish intervention: Spanish Spanish Spanish Spanish intervention:— Spanish intervention: 
(2006) Spanish and TOPPS (g = .82**) intervention: intervention: intervention: _ WLPB-Spanish WLPB-Spanish Listening 
English intervention English intervention: WLPB Spanish IDEL ORF | ~=WLPB-Spanish Passage Comprehension 

CTOPP (g = .38) LWID (g = (g = .41) Picture Comprehension (g = .23) 
.60**) IDEL ORF 2 Vocabulary (g  (g = .42) English intervention: 
WLPB Spanish (g = .28) = -.14) English intervention: WLPB-English Listening 
WA (g = .45) — English WLPB-Spanish VWLPB-English Comprehension 
Spanish word intervention: Verbal Passage (g = -.22) 
reading fluency* DIBELS ORF! = Analogies (g = | Comprehension 
(g = .48*) (g = -.39) .33) (g = .06) 
Domain average (g DIBELS ORF 2 English 
= .51*) (g = .27) intervention: 
English WLPB-English 
intervention: Picture 
WLPB-English Vocabulary (g 
LWID (g = .13) =-.17) 
WLPB-English WA WLPB-English 
(g = .15) Verbal Analogy 
TOWRE* (g =-.11) 
(g= Al) 


el 


(continued) 


vl 
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Table 3. (continued) 


Phonics/word Passage reading Vocabulary/oral Reading Reading 
Study PA reading fluency language cloze comprehension LC 
Begeny, Ross, — _— GORT fluency —_ — GORT — 


Greene, Mitchell, & (g = .95) comprehension 


Whitehouse (2012) (g = 1.00**) 
Denton, Anthony, — Read Well: —_ —_— Read Well: _ —_ 
Parker, & Hasrouck WRMT LWID WRMT Passage 
(2004) (g = .40) Comprehension 
WRMT WA (g = .18) 
(g = .33) Modified Read 
Modified Read Naturally: 
Naturally: WRMT Passage 
WRMT LWID Comprehension 
(g = —.06) (g = .15) 
WRMT WA 
(g = -.13) 
Lovett et al. (2008)° CTOPP Blending WRMT LWID —_— — WRIT Passage —_— — 
(g = .59*) (g = .00) Comprehension 
WRMT WA (g = .10) 
(g = .45) 
WRAT Reading 
(g = 33) 
Wanzek & Roberts _ Word study _ Word study Word study Word study Word study intervention: 
(2012) intervention: intervention: intervention: intervention: — VVIIII Listening 
WY) LWID GMRT W] Ill Passage GMRT Reading Comprehension 
(g = .38) Vocabulary (g¢ |= Comprehension Comprehension (g=.41) 
WJIIL WA = 59) (g =-.01) (g = —.62) Comprehension 
(g = 1.09**) Comprehension Comprehension Comprehension intervention: 
Domain average intervention: intervention: intervention: WVJIIl Listening 
(g = .73) GMRT WJ Ill Passage GMRT Reading Comprehension 
Comprehension Vocabulary (g © Comprehension Comprehension (g = -.42) 
intervention: = -.03) (g =-.21) (g = -.53) Responsive intervention: 
WII LWID Responsive Responsive Responsive W5III Listening 
(g=.13) intervention: intervention: intervention: Comprehension 
WJIII WA (g = .49) GMRT WJ Ill Passage GMRT Reading (g = .93*) 
Responsive Vocabulary (g © Comprehension Comprehension 
intervention: = —.05) (g =-.12) (g = -.33) 
WJII| LWID 
(g = .12) 


WIlll WA (g = .12) 
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Table 3. (continued) 


Phonics/word Passage reading Vocabulary/oral 


Study PA reading fluency 


Vaughn et al. (2011) —_— Individual _ 

intervention: 

WJIIT LWID (g = 
.18) 

W5JIIL WA (g = 
-.02) 

Standardized 
intervention: 

WJIT LWID (g = 
.23) 

W5JIIL WA (g = 
-.01) 


language 


Reading Reading 
cloze comprehension 


Individual — 
intervention: 
WJ Ill Passage 
Comprehension 
(g = -.05) 
Standardized 
intervention: 
WY} Ill Passage 
Comprehension 


(g = .26) 


LEG 


Note. DIBELS = Dynamic Indicators of Basic Early Literacy Skills; PSF = Phoneme Segmentation Fluency; ORF = Oral Reading Fluency; RD = researcher developed; PA = phonological 
awareness; LC = listening comprehension; WJ Ill = Woodcock Johnson (3rd ed.; Woodcock, McGrew, & Mather, 2001); LWID = Letter Word Identification; WA = Word Attack; CTOPP 
= Comprehensive Test of Phonological Processing (Wagner, Torgesen, & Rashotte, 1999); WRMT = Woodcock Reading Mastery Test (Woodcock, 1987); TOPPS = Test of Phonological 
Processing in Spanish (Francis et al., 2001); WLPB = Woodcock Language Proficiency Battery (Woodcock, 1991); IDEL = Indicadores Dinamicos del Exito en la Lectura (Good, Bank, & 
Watson, 2003); GMRT = Gates-MacGinitie Reading Test (MacGinitie, MacGinitie, Dreyer, & Hughs, 2006); GORT = Gray Oral Reading Test (Bryant, Shih, & Bryant, 2009); TOWRE = Test 


of Word Reading Efficiency (Torgesen, Wagner, & Rashotte, 1999). 
*Researcher-developed measure. 


’Studies had high attrition; either overall attrition or differential attrition effect sizes should be interpreted with caution. 


‘The effect size is a mean composite of the two phonics decoding measures. 
*p < .05.**p < 01. 
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the measures used: (a) reading cloze passage 
performance (e.g., Woodcock Reading Mas- 
tery Test (Woodcock, 1987)), (b) reading pas- 
sages with multiple-choice questions, and (c) 
listening comprehension. Outcomes for mea- 
sures in English and, if included in the study, 
primary language measures are reported. 

For each domain, we report both standard- 
ized and researcher-developed measures that 
were administered in the studies. Researcher- 
developed measures are denoted in Table 3. 
Given the small number of researcher-devel- 
oped measures, we include these in the ranges 
and median effect sizes that are reported. For 
each domain, we report the effect size range 
as well as the median effect size. We pur- 
posely do not report the mean effect size 
because of the variation in both the features 
of the studies and the characteristics of the 
interventions. As a representative effect size 
for the domain, we believe the median better 
preserves these variations than the mean, 
which by definition integrates this variation in 
the single score estimation process. 

Two studies (Vaughn, Cirino, et al., 2006; 
Vaughn, Linan-Thompson, et al., 2006) pro- 
vided interventions to students in Spanish. For 
each of the domains, it is important to con- 
sider that the students receiving the Spanish 
intervention were learning to read in their pri- 
mary home language with the intention of 
then transitioning to read in English, whereas 
students receiving the intervention in English 
were learning to read in a second language. In 
these studies, reading outcomes were assessed 
in Spanish, which we report. None of the other 
studies reported outcomes in the students’ pri- 
mary language. 


PA. Seven studies measured PA (Lovett et al., 
2008; O’Connor et al., 2010; Solari & Gerber, 
2008; Vadasy & Sanders, 2010; Vaughn, 
Cirino, et al., 2006; Vaughn, Linan-Thomp- 
son, et al., 2006; Vaughn, Mathes, et al., 
2006). Not surprisingly, all of these studies 
save for Lovett et al. (2008) targeted students 
in kindergarten or first grade. Effect sizes 
ranged from —0.74 to 1.24 with a median of 
0.59. Significant effect sizes ranged from 0.58 
to 1.24 with a median of 0.86. 


O’Connor et al. (2010) found significant 
effects for PA (g = 0.91), which was the main 
skill targeted in the intervention for this study. 
Vadasy and Sanders (2010) also found signifi- 
cant effects for PA (g = 0.93) for an interven- 
tion that included multiple components of 
reading, and the grades in this study, 2 through 
8, suggest other outcomes were of greater 
importance. In contrast, Solari and Gerber 
(2008) did not find significant effect sizes on 
PA measures. However, the treatment condi- 
tion in this study focused on listening compre- 
hension and the control group received a 
PA-only intervention, so the lack of effect on 
PA for the treatment group is not surprising. In 
the Vaughn studies (Vaughn, Cirino, et al., 
2006; Vaughn, Linan-Thompson, et al., 2006; 
Vaughn, Mathes, et al., 2006), effect sizes 
were significant for students who received the 
Spanish intervention, but results were mixed 
for the English interventions, as shown in 
Table 3. 


Phonics/word reading. Ten of the studies 
included outcome measures of word reading 
or decoding; most often, word identification 
and word attack (i.e., ability to decode pho- 
netically regular pseudowords). Effect sizes 
ranged from —0.19 to 1.09 with a median of 
0.33. Six studies had significant effect sizes 
ranging from 0.48 to 1.09 with a median of 
0.61. In three studies, significant effect sizes 
were found for phonetic decoding (Gunn 
et al., 2000; Vaughn, Linan-Thompson, et al., 
2006; Wanzek & Roberts, 2012), and in one 
study, a significant effect size was found in a 
combined measure of word identification and 
word attack (Vadasy & Sanders, 2010). Typi- 
cally, the effect sizes were larger for word 
attack than for word identification. In addition 
effect sizes tended to be smaller for interven- 
tions that were conducted with older students 
(second grade and above), except for the word 
study intervention used in the Wanzek and 
Roberts (2012) study, where the effect sizes 
ranged from 0.38 to 0.93. 


Fluency. Seven studies included measures of 
passage reading fluency as an outcome (Beg- 
eny et al., 2012; Gunn et al., 2000; Vadasy & 
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Sanders, 2010; Vaughn et al., 2011; Vaughn, 
Cirino, et al., 2006; Vaughn, Linan-Thomp- 
son, et al., 2006; Vaughn, Mathes, et al., 
2006). In each of these studies, fluency was 
measured using a l-min timed passage and 
was scored as number of words read correctly, 
except the study by Begeny et al. (2012), 
which used the Gray Oral Reading Test 
(GORT; (Bryant, Shih, & Bryant, 2009)) flu- 
ency subtest to measure fluency. On the 
GORT fluency subtest, the amount of time it 
takes to read a passage is recorded and a rate 
computed. On the fluency measures, effect 
sizes ranged from —0.39 to 0.95 with a median 
of 0.28. Significant effect sizes were found in 
only two of the studies. In the Vadasy and 
Sanders (2010) study, the effect size was 0.90 
for kindergarten students, and in the Vaughn, 
Linan-Thompson, et al. (2006) study, the 
effect size was 0.78 on the Spanish measure of 
reading fluency. In general, we did not find 
significant effects in passage reading fluency 
for English learners at risk or with learning 
disabilities who were taught in English. 


Vocabulary and oral language. Vocabulary and 
oral language were measured as an outcome 
in only four studies (Vaughn, Cirino, et al., 
2006; Vaughn, Linan-Thompson, et al., 2006; 
Vaughn, Mathes, et al., 2006; Wanzek & Rob- 
erts, 2012). Effect sizes ranged from —0.59 to 
0.78 with a median of —0.05. In only one study 
(Vaughn, Mathes, et al., 2006) was the effect 
size statistically significant. In three Vaughn 
et al. studies (Vaughn, Cirino, et al., 2006; 
Vaughn, Linan-Thompson, et al., 2006; and 
Vaughn, Mathes, et al., 2006), vocabulary and 
oral language were measured using the Pic- 
ture Vocabulary and Verbal Analogies subtests 
of the Woodcock Language Proficiency Bat- 
tery needs citation. Across the three studies 
and these two measures, the only significant 
effect was for English Verbal Analogies for 
first-grade English learners who received the 
intervention in English (g = 0.78); however, 
the domain average for vocabulary in this 
study was not significant. For students in 
fourth grade, across three types of interven- 
tions (word study focused, comprehension 
focused, and responsive based on individual 


need), no significant differences were found 
on vocabulary (Wanzek & Roberts, 2012). 
Thus, effect on vocabulary was minimal 
across the set of studies. 


Reading cloze. Reading cloze measures were 
used in eight of the studies (Denton et al., 
2004; Lovett et al., 2008; Vadasy & Sanders, 
2010; Vaughn et al., 2011; Vaughn, Cirino, 
et al., 2006; Vaughn, Linan-Thompson, et al., 
2006; Vaughn, Mathes, et al., 2006; Wanzek 
& Roberts, 2012). Typically, this skill was 
measured using the Passage Comprehension 
subtest from one of the Woodcock batteries. 
Effect sizes ranged from —0.21 to 0.88 with a 
median of 0.22. Significant effect sizes ranged 
from 0.47 to 0.88 with a median of 0.83. In 
two of the Vaughn studies (Vaughn, Linan- 
Thompson, et al., 2006; Vaughn, Mathes, 
et al., 2006) significant effects were found for 
the reading cloze measure for first-grade stu- 
dents. For the Spanish intervention, Spanish 
passage comprehension was significant at 
0.88, and for the English intervention, English 
passage comprehension was significant at 
0.83. In the Vadasy and Sanders (2010) study, 
effect sizes were significant on reading cloze 
for kindergarten students (g = 0.47). 


Reading comprehension. Reading comprehen- 
sion was measured in only two studies (Beg- 
eny et al., 2012; Wanzek & Roberts, 2012). 
Effect sizes ranged from —0.62 to 1.00, with a 
median of —0.48. A significant effect size was 
found on the GORT reading comprehension 
measure in the Begeny et al. (2012) study (g= 
1.00), which provided a reading fluency 
intervention. 


Listening comprehension. Five studies mea- 
sured listening comprehension, with effect 
sizes ranging from —0.42 to 2.34 and a median 
of 0.50. The pattern of findings is interesting. 
For upper elementary students, Wanzek and 
Roberts (2012) found a significant positive 
effect of 0.93 when the intervention was tai- 
lored to the student’s skill profile but nonsig- 
nificant impacts when a _ one-size-fits-all 
intervention was used. This result reflects a 
promising area for future research. For the 
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kindergartners, Solari and Gerber (2008) 
found significant positive effects for both 
their own measure of listening comprehension 
and the Woodcock Story Retell measure. Only 
one other effect was significant, for the Span- 
ish reading intervention by Vaughn, Linan- 
Thompson, et al. (2006). 


Moderating Variables 


To examine the specific features of the inter- 
vention that may have moderated intervention 
outcomes, we ran regression analyses using 
the average unweighted effect size as the 
dependent variable and group size (individual 
or small group), minutes of intervention (as a 
continuous variable), and personnel delivering 
the intervention (research personnel or school- 
based personnel) as the independent variables. 
In studies with more than one intervention, 
we included in the analysis the intervention 
that we determined the authors hypothe- 
sized would have the strongest effect. This 
was done to ensure that all contrasts were 
independent, an assumption for ordinary 
least squares regression. Results indicated 
that for each regression analysis, no signifi- 
cant relationship was found between the 
potential moderator variable (group size, 
minutes of intervention, or personnel deliv- 
ering the intervention) and the intervention 
outcomes. 


Discussion 


In this review, we examined the characteris- 
tics and outcomes of intervention studies that 
included data on English learners who were at 
risk for reading difficulties or had been identi- 
fied as having a reading disability. We located 
12 studies conducted since 2000 that used an 
RCT and met our criteria. The number of 
studies is dramatically smaller than the num- 
ber of high-quality reading intervention stud- 
ies that have been conducted with native 
English speakers over the same time period 
(see Edmonds et al., 2009; Solis, 2012; Wan- 
zek & Vaughn, 2007; and Wexler, Vaughn, 
Roberts, & Denton, 2010, for syntheses on 
reading interventions for non-English learners) 


but compares favorably to periods prior to 
2000 that addressed interventions with Eng- 
lish learners found by Gersten and Baker 
(2000). 

Unfortunately, given the limited sample of 
studies, the substantial amount of variation in 
the ages of the participants and types of inter- 
ventions conducted, and the variations in 
measuring outcomes, it is difficult to deter- 
mine patterns across the studies that would 
help identify potentially relevant trends. In 
this discussion, we summarize our findings 
illustrating patterns where possible, discuss 
the implications of our findings, and provide 
directions for future research. Regarding the 
reading interventions in Spanish, it is impor- 
tant to take into account that these interven- 
tions are, in a way, different from the English 
interventions, because in the former the stud- 
ies used reading in the native language as an 
approach to providing reading support to 
struggling English learners, whereas in the 
latter the studies explored the impact of an 
intervention in a second language on student 
reading performance. 


Features of the Intervention Studies 
and Their Relationship to Impacts 


Our review revealed a large variability in how 
English learners were identified and defined 
across studies. This trend has been an issue for 
many years, with frequent requests for more 
consistency in how English learners are iden- 
tified in research studies. For example, in 
some of the studies, the school designation of 
English learners was used, whereas in other 
studies, researchers used only a home lan- 
guage survey and not an individually adminis- 
tered oral language test. 

Given the heterogeneity of the English learner 
population, in terms of both language proficiency 
and academic achievement, interventions that 
may be effective for one group of English learn- 
ers may not be effective with others (August & 
Shanahan, 2006). Thus, including specific lan- 
guage proficiency information as well as aca- 
demic proficiency is important in intervention 
studies so readers can understand “for whom” 
the described intervention is effective (Klingner 
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et al., 2006). The practice of providing greater 
specification of the student samples is particu- 
larly critical for English learners, given the 
importance of language factors on achievement 
outcomes. 

Across the 12 studies in our sample, only 
two reported an analysis of the differential 
effect of English language proficiency on out- 
comes. In the O’Connor et al. (2010) study, 
the language proficiency level of participants 
as measured by the California English Lan- 
guage Development Test did not have an 
impact on how well kindergarten students 
responded to the intervention. This is consis- 
tent with previous research with English 
learners on the weak association between 
language proficiency and reading growth in 
the early grades, particularly on foundational 
measures of reading, such as phonemic 
awareness and decoding (Chiappe, Siegel, & 
Wade-Woolley, 2002; Gersten et al., 2007). 
However, it differs from more recent research 
that found that language proficiency appears 
to affect early reading skills (D. Baker et al., 
2013; Kieffer, 2008). On the other hand, in the 
Lovett et al. (2008) study, which targeted 
English learners in Grades 2 through 8, the 
finding was that students who began the inter- 
vention with higher levels of language profi- 
ciency responded more positively to the 
interventions based on measures of phonemic 
blending and passage comprehension than 
students who began the intervention at lower 
levels of language proficiency. These findings 
are consistent with evidence that English lan- 
guage proficiency has an impact on student 
outcomes particularly in the upper elementary 
grades (see Geva & Farnia, 2012; Kieffer, 
2010). 

More research is needed to determine how 
varying levels of English language profi- 
ciency affect the impact of an intervention. In 
particular, it may be that growth in basic read- 
ing skills (decoding and literal comprehen- 
sion) is not related to higher levels of English 
language skill but that growth on higher-level 
skills, such as comprehension, is. Moreover, it 
may be that students who are technically 
exited from English learner status (often 
called former English learners; e.g., Parrish 


et al., 2006), but may not have developed the 
necessary academic English to be successful 
in school, are noticeably absent from the 
intervention studies in this review. That is, the 
English learner sample in these studies may 
be lower in English language proficiency than 
the population of English learners currently in 
American schools. Given this fact and the fact 
that most studies did not report levels of Eng- 
lish language proficiency of the sample, we 
caution making extrapolations or generaliza- 
tions from this small set of studies. 


Intensity Factors of Group Size, 
Duration, Personnel, and Quality 


Our review indicated there was large variation 
across interventions in terms of group size, 
minutes of instruction, and personnel deliver- 
ing the instruction. As in most meta-analyses, 
it is hard to disentangle length of intervention 
from numerous other factors. As discussed 
earlier, the nature of the counterfactual varied 
dramatically across studies, ranging from pro- 
viding no intervention at all to providing the 
school’s typical reading intervention. Thus, 
although the moderator analyses showed no 
significant role in predicting effect size, it 
does not mean that these intensity factors are 
not relevant. 

One hallmark of Tier 2 interventions is 
delivery of the intervention in small groups 
(Gersten et al., 2007; Vaughn, Wanzek, Wood- 
ruff, & Linan-Thompson, 2007). Research has 
indicated that reasonably homogeneous small 
groups are often effective for delivering 
instruction, particularly for students at risk or 
with learning disabilities (Ehri et al., 2001; 
Elbaum, Vaughn, Hughes, & Moody, 1999; 
Vaughn et al., 2003). Eleven of the interven- 
tions reviewed used small-group instruction; 
two applied the intervention with students indi- 
vidually (Begeny et al., 2012; Vadasy & Sand- 
ers, 2010). When we analyzed group size as a 
moderating variable, we found there was no 
significant difference between individualized 
intervention and small-group interventions. It 
stands to reason that small-group interventions 
of three to five students might be more effec- 
tive than either individually delivered interven- 
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tions or interventions delivered to six or more 
students, because English learners have more 
opportunities to practice the skill they are 
working on as well as their English language 
proficiency with their peers and the teacher, 
and the small-group settings provide more 
opportunities for this than they would get if 
they were in a large group (D. Baker & Kosty, 
2012; Gersten & Jiménez, 1998). However, the 
results of this research synthesis do not demon- 
strate consistent, significant positive impacts or 
even consistently positive effects. 

Interventions varied substantially in terms 
of the amount of instructional time provided. 
However, length of intervention did not pre- 
dict magnitude of effect as the moderator 
analysis indicated. One reason that minutes of 
intervention may not have influenced effect 
sizes is that although intervention treatment is 
longer, so is the instruction provided to the 
students in the control group. In addition 
shorter interventions tended to focus on just 
one or two reading outcomes and often mea- 
sured only these specific outcomes, which 
may have accounted for larger effect sizes for 
these studies. For example, studies that 
focused on kindergarten students targeted 
only foundational reading or prereading skills 
(1.e., O’Connor et al., 2010; Solari & Gerber, 
2008). In contrast, studies that focused on 
multiple components of reading—as many 
believe is most appropriate for Grades | and 
up—demonstrated quite mixed results. For 
the intermediate grades, the one study that tai- 
lored interventions to students’ skill profiles 
(Wanzek & Roberts, 2012) tended to be much 
more effective than those with a “one-size- 
fits-all” approach. This seems a promising 
direction to pursue for future response-to- 
intervention research with English learners. 
However, one should not overgeneralize from 
the one study. Although it is important to tar- 
get interventions to the specific skills or profi- 
ciencies that students are lacking, at some 
point in time, it is also necessary to help stu- 
dents orchestrate the various components of 
reading (D. Baker, Stoolmiller, Good, & 
Baker, 2011). To date, there is little response- 
to-intervention research that addresses the 
issue of orchestration. 


For older students in middle school, min- 
utes of instruction did not appear to have an 
impact on the results. For example, in the 
Vaughn et al. (2011) study, English learners in 
middle school received a full year of a Tier 3 
reading intervention for 50 min a day, approx- 
imately 8,000 min of instruction. This inter- 
vention did not yield significant effects, 
suggesting that older English learners who 
have significant reading difficulties may need 
longer and more intensive interventions than 
younger English learners, a finding that would 
be consistent with findings from intervention 
studies with English-only students (Biancarosa 
& Snow, 2004; Torgesen et al., 2001). 

Our moderator variable analysis also indi- 
cated that there were no differences between 
researcher-delivered interventions and the 
school personnel—delivered interventions. We 
were surprised but encouraged that there were 
not differences, because this indicates that, 
with adequate training, interventions can be 
delivered by school-based personnel with 
similar impact. 

A characteristic common across all studies 
was delivery of instruction. All the studies 
reported using explicit and systematic instruc- 
tion. Although the level of explicitness and 
systematicity varied across studies depending 
on the interventions used, most studies used 
the following routines and general progres- 
sion to deliver systematic and explicit instruc- 
tion: Teachers modeled and demonstrated, 
teachers led guided practice, students received 
many opportunities to practice the activities 
on their own, and review of previously learned 
content and material was regularly incorpo- 
rated throughout the intervention. This find- 
ing supports a substantial body of evidence on 
the benefits of using systematic and explicit 
instructional routines, particularly with stu- 
dents who are at risk or who have a reading 
disability, regardless of their language status 
or demographic characteristics (Edmonds 
et al., 2009; Ehri et al., 2001; Gersten et al., 
2007; Swanson, 1999). 

The nature of the counterfactual also may 
play arole in the outcomes of the interventions. 
However, consistent information regarding the 
control group instruction was not found across 


Downloaded from ecx.sagepub.com by guest on August 12, 2015 


Richards-Tutor et al. 


21 


studies. Also, the number of students in the 
control group who received intervention and 
for how many minutes the students received 
the intervention were not consistent within 
studies. This is not surprising given the nature 
of school-based research. Most studies did pro- 
vide information regarding the core instruction 
provided to all students and did thorough 
observations of this instruction. 


Outcomes of the Interventions 


Although many of the studies measured a vari- 
ety of outcomes across all areas of reading, 
interventions that focused on improving foun- 
dational skills, such as PA and phonics, with 
younger students in kindergarten and _ first 
grade obtained better and more consistent 
effects than other outcomes, such as those 
interventions that focused on improving vocab- 
ulary and comprehension. Typically, the effects 
of intervention on older English learners 
(fourth grade and above) were minimal except 
for a few measures across the four relevant 
studies; only one study (Wanzek & Roberts, 
2012) showed significant effects in listening 
comprehension for older students. As stated 
previously, the intervention with significant 
effects was tailored to each student individu- 
ally, thereby being highly “responsive” to the 
needs of each individual student. In this man- 
ner, the intervention had the characteristics of a 
model Tier 3 intervention compared to a more 
proscribed Tier 2 approach. The positive find- 
ings in this study suggest, particularly for older 
English learners perhaps, that an intervention 
based on individual student patterns of perfor- 
mance might be more effective than even 
highly intense interventions provided in 
roughly the same way to all at-risk students. 
We could not locate any studies that tar- 
geted vocabulary specifically, and only four 
studies measured vocabulary as an outcome. 
This was surprising given that vocabulary 
plays a major role in the reading development 
of all students but is particularly important for 
English learners, as suggested by Gersten and 
Baker (2000) more than a decade ago and by 
Jiménez, Garcia, and Pearson (1996) almost 
two decades ago. Even more surprising is that 


only four studies have been published on 
vocabulary interventions for English learners 
in the past three decades (Carlo et al., 2004; 
Cena et al., 2013; Perez, 1981; Vaughn- 
Shavuo, 1990), and none of these studies 
focused specifically on English learners who 
were at risk or had learning disabilities. 

We are encouraged by the findings in the 
Solari and Gerber (2008) study, which showed 
significant effects for listening comprehen- 
sion on both researcher-developed and stan- 
dardized measures of listening comprehension 
in kindergarten for English learners when they 
were provided an intervention with a strong 
listening comprehension component. For the 
most part, many of the interventions paid only 
cursory attention to developing either oral or 
written language skills as part of the interven- 
tion approach. For many, this seems counter- 
intuitive because of language demands in 
academic settings. For others, it is understand- 
able because of the challenges associated with 
intervening systematically in language. 

In the Solari and Gerber (2008) study, lan- 
guage skills were taught along with academic 
skills to support later reading comprehension. 
It provides an important example of how lan- 
guage instruction can be incorporated into 
English learner research. The study is also 
noteworthy because it demonstrated that even 
students who are not yet proficient in English 
benefit from an intervention focusing on both 
language and reading development. Given the 
increased emphasis on academic language 
and academic vocabulary in the Common 
Core State Standards, recent publications on 
effective instruction for English learners (e.g., 
S. Baker et al., 2014), and research indicating 
that students with poor reading comprehen- 
sion often have language deficits (Catts, 
Adolf, & Ellis Weismer, 2006; Catts, Compton, 
Tomblin, & Bridges, 2012), we anticipate—or at 
least hope for—a surge of studies examining 
vocabulary and academic language interven- 
tions for this population. 

Finally, it also was striking to us that so 
many of the interventions provided were iden- 
tical to those provided to native speakers. 
Although this makes perfect sense in kinder- 
garten and early first grade, afterward, we 
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wonder if more innovative intervention curri- 
cula that have a heavy language component 
might produce stronger effects than the cur- 
rent set of studies. 


Implications for Future Research 


In our search for studies for this review, we 
found 12 studies that used an RCT and met our 
inclusion criteria. More experimental studies 
ought to be conducted to determine what mal- 
leable factors have a significant effect on Eng- 
lish learners’ academic performance. However, 
the pace of studies is improving, and the num- 
ber of studies conducted since 2000 is much 
greater than similar time periods prior to 2000. 
In terms of implications for future research, we 
recommend that researchers (a) focus on the 
individual differences in English learners, (b) 
consider development of interventions that 
focus on language and vocabulary and mea- 
sures that capture language comprehension, 
and (c) include the calculations of an “effort 
variable” to be able to compare interventions. 

There are scarce studies that focus on Eng- 
lish learners at risk for reading disabilities and 
even fewer studies that disaggregate the data 
by student language proficiency. English 
learners are a very heterogeneous group of 
students. They vary in terms of language pro- 
ficiency, academic achievement, and the myr- 
iad predictor variables that may have an 
influence on growth and performance, such as 
poverty status and proficiency in their pri- 
mary language. Future research should inves- 
tigate interventions for English leaners at 
varying language proficiency levels, includ- 
ing students who are technically exited from 
English learner status. Evaluating how indi- 
vidual differences in language proficiency 
influence intervention outcomes will allow 
the field to refine interventions to better meet 
the needs of these students. 

In addition, there is clearly a need to exam- 
ine the effect of interventions that focus on lan- 
guage development and vocabulary as a core 
component for English learners at risk and 
those who have learning disabilities. We found 
very few studies that included a vocabulary or 
language development component and even 


fewer that measured this domain as a pretest or 
outcome. Those that did measure vocabulary 
used typical standardized measures that may 
not be able to capture the growth students are 
making in the context of a particular interven- 
tion. Better language and vocabulary measures, 
and better measurement development proce- 
dures that can be used in the context of specific 
studies, are badly needed to more accurately 
estimate the impact of interventions on lan- 
guage and vocabulary outcomes. 

We recommend that future intervention 
studies include the calculation of an inter- 
ventionist “effort variable” to guide practi- 
tioners in the allocation of resources to 
support struggling English learners. This 
effort variable might include minutes of 
instruction divided by number of students in 
a group to help the field learn more about 
what is the most effective amount of time 
and group size to obtain a desired effect. In 
addition, a better description of who is deliv- 
ering the intervention (e.g. a certified teacher, 
a research assistant, or a trained instructional 
assistant), and the amount of hours needed to 
train staff to deliver the intervention with 
fidelity, could help schools improve the sup- 
ports they provide English learners. The 
English learner population will continue to 
increase, and the achievement gap with non— 
English learners will not be reduced unless 
the effect of interventions for English learn- 
ers at risk or with learning disabilities is rig- 
orously evaluated, to better understand how 
to effectively support these students. 
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